Interactively Transferring CNN Patterns for Part Localization

This paper explores an interactive method to diagnose knowledge representations of a CNN, in order to use CNN knowledge to model object parts. Unlike traditional end-toend learning of CNNs that require numerous training samples, given a CNN pre-trained for object classification, we mine object part patterns from the CNN in the scenario of one/multi-shot learning. More specifically, our method uses very few (e.g. three) object images to summarize knowledge in conv-layers into a dictionary of latent activation patterns. For each object part, our method visualizes the latent patterns and asks users to manually assemble latent patterns related to the target part, so as to construct the object-part model. As a general solution, our interactive method was broadly applied to different types of neural patterns in experiments. With the guidance of human interactions, our method exhibited superior performance of part localization.1